A scalable elliptic solver with task-based parallelism for the SpECTRE numerical relativity code

Elliptic partial differential equations must be solved numerically for many problems in numerical relativity, such as initial data for every simulation of merging black holes and neutron stars. Existing elliptic solvers can take multiple days to solve these problems at high resolution and when matter is involved, because they are either hard to parallelize or require a large amount of computational resources. Here we present a new solver for linear and non-linear elliptic problems that is designed to scale with resolution and to parallelize on computing clusters. To achieve this we employ a discontinuous Galerkin discretization, an iterative multigrid-Schwarz preconditioned Newton-Krylov algorithm, and a task-based parallelism paradigm. To accelerate convergence of the elliptic solver we have developed novel subdomain-preconditioning techniques. We find that our multigrid-Schwarz preconditioned elliptic solves achieve iteration counts that are independent of resolution, and our task-based parallel programs scale over 200 million degrees of freedom to at least a few thousand cores. Our new code solves a classic black-hole binary initial-data problem faster than the spectral code SpEC when distributed to only eight cores, and in a fraction of the time on more cores. It is publicly accessible in the next-generation SpECTRE numerical relativity code. Our results pave the way for highly-parallel elliptic solves in numerical relativity and beyond.Comment: 24 pages, 18 figures. Results are reproducible with the ancillary input file

Worldtube excision method for intermediate-mass-ratio inspirals: scalar-field model in 3+1 dimensions

Binary black hole simulations become increasingly more computationally expensive with smaller mass ratios, partly because of the longer evolution time, and partly because the lengthscale disparity dictates smaller time steps. The program initiated by Dhesi et al. (arXiv:2109.03531) explores a method for alleviating the scale disparity in simulations with mass ratios in the intermediate astrophysical range ($10^{-4} \lesssim q \lesssim 10^{-2}$), where purely perturbative methods may not be adequate. A region ("worldtube") much larger than the small black hole is excised from the numerical domain, and replaced with an analytical model approximating a tidally deformed black hole. Here we apply this idea to a toy model of a scalar charge in a fixed circular geodesic orbit around a Schwarzschild black hole, solving for the massless Klein-Gordon field. This is a first implementation of the worldtube excision method in full 3+1 dimensions. We demonstrate the accuracy and efficiency of the method, and discuss the steps towards applying it for evolving orbits and, ultimately, in the binary black-hole scenario. Our implementation is publicly accessible in the SpECTRE numerical relativity code.Comment: 19 pages, 10 figure

Worldtube excision method for intermediate-mass-ratio inspirals: scalar-field model in 3+1 dimensions

Binary black hole simulations become increasingly more computationally expensive with smaller mass ratios, partly because of the longer evolution time, and partly because the lengthscale disparity dictates smaller time steps. The program initiated by Dhesi et al. [Phys. Rev. D 104, 124002 (2021)] explores a method for alleviating the scale disparity in simulations with mass ratios in the intermediate astrophysical range (10−4≲q≲10−2), where purely perturbative methods may not be adequate. A region (“worldtube”) much larger than the small black hole is excised from the numerical domain, and replaced with an analytical model approximating a tidally deformed black hole. Here we apply this idea to a toy model of a scalar charge in a fixed circular geodesic orbit around a Schwarzschild black hole, solving for the massless Klein-Gordon field. This is a first implementation of the worldtube excision method in full 3+1 dimensions. We demonstrate the accuracy and efficiency of the method, and discuss the steps toward applying it for evolving orbits and, ultimately, in the binary black-hole scenario. Our implementation is publicly accessible in the spectre numerical relativity code

SpECTRE

<p>SpECTRE is an open-source code for multi-scale, multi-physics problems in astrophysics and gravitational physics. In the future, we hope that it can be applied to problems across discipline boundaries in fluid dynamics, geoscience, plasma physics, nuclear physics, and engineering. It runs at petascale and is designed for future exascale computers.</p> <p>SpECTRE is being developed in support of our collaborative Simulating eXtreme Spacetimes (SXS) research program into the multi-messenger astrophysics of neutron star mergers, core-collapse supernovae, and gamma-ray bursts.</p&gt

Simulating magnetized neutron stars with discontinuous Galerkin methods

Discontinuous Galerkin methods are popular because they can achieve high order where the solution is smooth, because they can capture shocks while needing only nearest-neighbor communication, and because they are relatively easy to formulate on complex meshes. We perform a detailed comparison of various limiting strategies presented in the literature applied to the equations of general relativistic magnetohydrodynamics. We compare the standard minmod/Lambda Pi(N) limiter, the hierarchical limiter of Krivodonova, the simple WENO limiter, the HWENO limiter, and a discontinuous Galerkin-finite-difference hybrid method. The ultimate goal is to understand what limiting strategies are able to robustly simulate magnetized Tolman-Oppenheimer-Volkoff stars without any fine-tuning of parameters. Among the limiters explored here, the only limiting strategy we can endorse is a discontinuous Galerkin-finitedifference hybrid method